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ABSTRACT 



Faculty at the College of Veterinary Medicine, Cornell 



University, have developed a bibliographic database of articles to support 
problem based learning. The database, its role in the curriculum, and its 
effectiveness in guiding, without undermining, self -directed learning were 
studied, using the goals of problem based learning as a framework. Students' 
self-reports in the form of end-of -course student evaluations were used to 
learn how students use the database and perceive its impact on student 
learning. Responses of 3 cohorts of approximately 84 students each 
representing 2 courses were analyzed. Many students in both courses used the 
database and found it useful in addressing learning issues. The database did 
not eliminate the "search stress" students had described, but findings 
suggest that there are two types of search difficulty. One was related to the 
mechanics of obtaining and copying papers, and the other was centered on 
learning hot to cope with too much information and reading for new key 
concepts. On the whole, the goals of the database appear to have been met. A 
majority of students in both courses used the resource to access relevant 
primary literature, and the database helped students focus on a smaller 
collection of key articles rather than the huge results often found in a 
Medline search. (SLD) 
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Introduction 

Students and faculty often find the shift from a traditional curriculum to a 
problem based learning (PBL) curriculum difficult. In a problem based learning 
curriculum, students’ learning is prompted by and situated in real world problems 
encountered in the profession. Problem based learning is structured to help students: 1) 
learn important principles and key concepts; 2) develop their problem solving skills (the 
clinical reasoning process in particular), and 3) learn how to direct and manage their own 
learning (Barrows & Tamblyn, 1980; Barrows, 1988; Schmidt, 1993). The success of a 
problem based learning curriculum depends upon students taking responsibility for their 
own learning. Students, like professionals in the field, are expected to prioritize what they 
need to learn (“learning issues”), make choices about the resources they will consult, 
work collaboratively with colleagues, and organize their efforts to address learning issues 
in sufficient depth. 

It can be particularly challenging for students to develop skill and confidence in 
directing their own learning. Among the difficulties students experience with this 
transition are the demands of selecting appropriate literature to address their learning 
issues. In a study of students enrolled in a problem based learning physiotherapy 
program, Solomon and Finch (1998) identified 10 stressors that were described by at least 
one quarter of the students in a reflective journal kept during the first semester of their 
program. “Search stress” - difficulty with finding appropriate literature and inordinate 
amounts of time spent searching rather than studying - was mentioned by almost one 
third of their students. 

Faculty, too, may find it frustrating to watch students struggling and still missing 
valuable resources. On the one hand, educators wish to encourage self-directed learning, 
in keeping with a core philosophy and purpose of problem based learning. On the other 
hand, if students are spending disproportionate time and energy trying to find helpful 
literature or end up missing key references altogether, it can compromise another core 
goal of PBL - that students learn the scientific principles underlying the case or problem. 
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A challenge for PBL educators, then, is helping to strike the right balance so that students 
are both accessing the literature that will help them learn the subject matter and 
developing their self directed learning skills. 

To balance these two goals of problem based learning, faculty in the College of 
Veterinary Medicine at Cornell University have developed a bibliographic database of 
articles. This literature database, its role in the curriculum, and its effectiveness at guiding 
(without undermining) self-directed learning are investigated, using the goals of problem 
based learning as a framework. Students’ use of and response to the database in two 
different courses are examined. The main questions addressed are: How many students 
regularly use the database? What affect does the database have on students’ use of 
Medline and, consequently, a wider range of primary literature? How is the database 
rated in terms of its usefulness in addressing learning issues? What impact does the 
database have on students’ self-directed learning and “search stress”? What can be 
learned about the use of this database that would be helpful to those interested in 
implementing it in other courses? 

Principles Underlying the Design of the Veterinary College Literature Database 

In the fall of 1998, the Veterinary College Literature Database (VCLD) was 
piloted in two, large, full time interdisciplinary courses. Foundation Course II: Genetics 
and Development and Foundation Course IV: Host, Agent and Defense, (for a short 
description of the problem based learning approach used at Cornell, see Quinlan, 2000). 
The database was used again in both courses in fall 1999. In each of these courses, 
faculty wanted students to study from journal articles, rather than just textbooks. 
However, only the best students were successfully navigating Medline. Unless literature 
searches are narrowly focused, Medline yields unmanageable results for students. Most 
students were simply frustrated and overwhelmed by the large volume of results that 
Medline generated. Furthermore, the faculty associated with those courses has a number 
of useful articles that they wanted students to have easy access to, without spoon feeding 
them particular papers for particular cases. 

The Veterinary College Literature Database was developed as a catalog of 
references that faculty recommended on the basis of clarity, accuracy, relevance to the 
field, and up-to-date information. Review articles were favored over primary, empirical 
research reports. The intent was to offer a number of different articles that addressed 
course objectives. Key words were assigned to each article, allowing students to search 
the collection more easily. Referencing several papers on the same broad topic pushed 
students to evaluate and make choices on their own and/or to learn to skim for relevancy. 
Thus, students were still expected to direct their own learning and to develop the skills 
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involved in doing so. Once a reference is found in the database, students go to the 
journals on the library shelves to find, read, or photocopy selected articles. This structure 
was intended to ensure that students familiarize themselves with the journals. For articles 
that are not available in the library, photocopies are put on reserve. Since the database is a 
College-wide resource, there are articles referenced for more than one foundation course, 
totaling more than 700 references in the database. 

In Course II: Genetics and Development, a first year course addressing cell 
commitment and movement, morphogenesis and growth, oncogenesis, sex determination 
and early development, the focal disciplines are changing so rapidly that recent and 
current journal articles are primary resources in the course. However, in Course IV: 

Host, Agent and Defense, a second year course focusing on inflammation and infection, 
the immune system and immunopathology, bacteriology and mycology, parasitology, 
virology, antimicrobial therapy and epidemiology, there are nine suggested texts in 
addition to the database of journal articles. Thus, the primary literature is a supplement 
to other texts. 



Method and Data Sources 

For each course, students’ self reports in the form of end-of-course student 
evaluations were used to learn how students use the database and perceive its impact on 
student learning. Evaluation questions addressed whether the students used the VCLD, 
whether they used Medline, and the usefulness of the VCLD. Open-ended questions 
about the course resources (from the 1998 offering of Course II) also offered additional 
data on the impact of the literature database. The questions asked on the Course II and 
Course IV evaluations were worded differently, making direct comparisons between the 
courses more difficult. Across both years and both courses, reactions from three cohorts 
of approximately 84 students (Classes of 2001, 2002, 2003) were gathered and analyzed. 

Results 

The response rate for Course II in fall 1998 was 77 out of 84 students, or 92%. In 
1999, the response rate for Course II was 79 out of 83 students or 95%. The response rate 
for Course IV in fall 1998 was lower: 65%, with 55 out of 84 students responding. 
Because of the low response rate in Course IV in 1999 (44%), data from that course 
offering is not included in this analysis. 

In Course II, nearly all students who responded to the survey used the literature 
database (95% in 1998; 96% in 1999). In 1998 the VCLD received a mean rating of 3.89 
(on a scale of 1-5, ^inadequate; 5=excellent) as a tool in finding answers to students’ 
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learning issues. That rating was even higher (4.21) in 1999. The database, though, did 
not serve as a substitute for students’ use of Medline, since 44% of the students reported 
using Medline in addition to the VCLD in 1998 (29% in 1999). Most students (78% in 
1998; 83% in 1999) did not want Medline to replace the VCLD. In other words, the 
majority of students appreciated the value of the smaller database above and beyond what 
Medline does for them. 

In contrast, 12% of the 1998 respondents thought that students should use 
Medline instead of the VCLD. Seven students explicated that view in their comments, 
suggesting that the database compromised the self-directed learning principles espoused 
in the curriculum. One student wrote, 

I made our tutor group meet with the library staff for an Intro to Medline Searching 
[session] because, although a lot of effort went into preparing the VCLD, when we get 
into the real world we’re not going to be able to go see what the higher powers have put 
aside for us on our topic of interest. We're going to need to know how to make the best 
use of Medline. 

This small group of students saw searching as a key skill to be developed and, 
thus, wanted the experience to be most like “real life” rather than a sheltered 
environment. 

However, many students continued to voice “search stress” concerns. Some 
comments focused on the mechanics of sifting through articles under time pressure, or 
spending too much time and money photocopying. However, sixteen students made 
negative comments about the intellectual demands of reading the primary literature 
referenced in the database. Many students felt that the struggle to comprehend difficult 
texts was not worth it given the focus of the final exam. Some comments also suggested 
that students might need additional attention paid to their skill in reading primary 
literature, particularly in how to read that literature critically for its main points, without 
losing the forest for the trees: 

Inform students that the majority of the papers are optional. They are interesting to look 
at, but are very specific, and often hinder learning because students are more caught up in 
figuring out what a paper is trying to say than concentrating on the actual topics of 
learning for that week.... I think that many students (mostly early on) were exasperated 
by trying to ingest the big words of the scientific articles and became bogged down by 
this rather than gaining a full understanding of things that were central to the course. 

Finally, several students indicated that the sheer volume of available articles was 
overwhelming, asking faculty to sort the list of readings to reduce overlap and to focus on 
just the most important, required articles for the exam. 

While sixteen students made negative comments about the difficulty of sifting 
through the primary literature, three students cited the reading of primary literature as one 
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of the most enjoyable aspects of the course. Thirteen students either specifically praised 
the database (7) or said that the resources were “very good”, “adequate” or of “excellent 
quality.” (6) For example, “I thought the limited VCLD was appropriate for the course. It 
provided enough direction to know what we should be focusing on. I think Medline 
would be too varied.” And, “I used the VCLD because the papers on there were ones 
that we had and used. I used Medline a couple of times and it often gave me too many 
articles and not ones I would find useful.” Thus, a total of sixteen students spoke highly 
of the resources made more accessible through the literature database. 

In Course IV, there were no open-ended questions focused on the database and, in 
both years, very few comments were offered elsewhere on the evaluation about the 
database. The course designers’ impression was that students responded well to the 
database. In fact, 47 of the 55 students (85%) rated the database as “adequate” or better in 
1998. Thirty-six rated it as “very good” or “excellent” in identifying resources useful to 
one’s learning issues (on a scale of 1-5, l=inadequate, 5=excellent. Mean rating = 3.65). 
In the past, students had been encouraged to read the primary literature by searching 
Medline, but “only the most exceptional students did that,” according to a course 
specialist. With the database, relevant journal articles were more easily found and used. 
Having better access to appropriate literature was seen to “level the playing field” in that 
more students went to the primary literature than without the database. Indeed, the 
course evaluation showed a high rating (3.29) of students using the VCLD, while 
considerably fewer used Medline regularly (1.74). 

Discussion 

Many students in both courses used the Veterinary College Literature Database 
and most found it useful in addressing learning issues. The database did not eliminate 
“search stress,” particularly in Course H. Student comments, though, suggest that there 
may be two types of “search” difficulty. One type of complaint focused on the 
mechanics of actually obtaining, borrowing and photocopying papers. Students with this 
type of search stress wanted the “grunt work” removed by distributing copies or 
providing more copies. A second kind of stress was centered on learning how to cope 
with too much information and reading for new, key concepts. For those students, the 
anxiety seems to come from having to make difficult decisions for themselves about how 
to prioritize their learning issues and how to best address them within a limited time. It is 
possible, though, that some of the “mechanics” concerns could be eliminated if students 
had a better grasp of the intellectual demands of the self-directed learning and prioritizing 
process. In other words, if students skimmed and prioritized better, they might not feel 
obligated to photocopy a dozen papers, thus reducing complaints about the inefficiency 
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and expense of photocopying articles. Only a few students - those who were apparently 
comfortable with the ambiguity of the learning situation and appreciated the principles of 
self-directed learning - felt that too much guidance was given. 

In Course IV, there were fewer comments to analyze, making it more difficult to 
compare student responses across the two courses. While faculty in Course IV thought 
that the database was received positively, 46% of the students rated the database as “very 
good” or “excellent.” Mixed student comments in Course II would suggest ongoing 
search stress, despite the additional guidance provided by the database. However, in that 
course, 67% rated the VCLD as “very good” or “excellent.” These differences in ratings 
may be because fewer students used the database in Course IV (only 58% gave it a rating 
of 4 or 5 in response to the question: “I used the database on a regular basis”). This result 
is difficult to interpret because the answer parameters did not match the wording of the 
question. More precise descriptors of the 1-5 scale would improve the accuracy of the 
results. However, 95% of the students in Course II used the database, though we don’t 
know how regularly they used it. Thus, the lower ratings of the database in Course IV 
may be because fewer students used that resource. Certainly, the database played a more 
central role in Course II than it did in Course IV. 

Students’ patterns of use in Block II versus Block IV may be explained by 
different emphases placed on the literature database in each of the courses. Perhaps 
faculty in Course II consistently stressed the importance of the articles in the database, 
while the faculty in Course IV, may have given the database less priority. 

On the whole, the goals of the database seem to have been met. A majority of 
students in both courses used the resource to access relevant primary literature. This fact 
alone is worth noting, as many faculty question how much students use supplementary 
materials. 

The database also helped to focus students on a smaller collection of key articles, 
which were more closely related to the course objectives, rather than the huge results 
often found on a Medline search. As one student put it, narrowing the literature helped 
“insure that group members came in [to tutorial] to discuss similar things.” In Course IV, 
the database may have helped students to access the primary literature more regularly 
than they had with Medline alone. Many students’ continued use of Medline in Course II 
indicates that the database did not detract significantly from students using a wider range 
of resources. The students’ open-ended comments about their experience of the database 
shows that they were still expected to direct their own learning, albeit within a more 
constrained environment. It may be neither possible nor desirable to reduce all “search 
stress”. Learning more about “search stress” and the process of becoming better at self- 
directed learning in ambiguous learning situations may help problem based learning 
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educators to find appropriate ways to support students as they learn to adjust to a problem 
based learning curriculum. 



References 

Barrows, H.S. (1988) The tutorial process. Springfield, IL: Southern Illinois 
University School of Medicine. 

Barrows, H.S. & Tamblyn (1980). Problem based learning: An approach to 
medical education. New York: Spring- Verlag. 

Quinlan, K.M. (2000). PBL at Cornell University’s College of Veterinary 
Medicine: Implications for Undergraduate Education. PBL Insight. 3(1) . pp 1,4-5. 

Schmidt, HG. (1993). Foundations of problem-based learning: Some explanatory 
notes. Medical Education, 27, 422-32. 

Solomon, P. and Finch, E. (1998) A qualitative study identifying stressors 
associated with adapting to problem-based learning. Teaching and Learning in Medicine , 
10 (2), 58-64. 



Paper discussed at a roundtable at the Annual Meeting of the American Educational Research Association 

New Orleans, LA, April 2000. 




9 




AERA 



U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 

Educational Resources Information Center (ERIC) 

REPRODUCTION RELEASE 

(Specific Document) 







— - — ■ wa a ■ g w ■«« 

Title: g|r)k«OA+U fc'cxW- £ci| '■ fin £\faluad - tots 0-f- ^ t, 'tenth ^ 
^0 SappoVV - EoSfA /^drniOq 


- Authors): ra^kleevx fY\ . 


Wrvlflyx ° 


Corporate Source: 

li. REPRODUCTION RPI pace. 


Publication Date: 



monthly abstract Journal of the ERIC ayston^sources ifaSaSwi (RIB community, documents announced in the 

am l!! 6 ^ iC media ' ,nd thiwShtte ERIC DocumertRep^uSwiSenri^^EDRS? 8 Ct^it^ (jiven^ re P? duoed 
reproduction release is granted, one of the following notices is affixed to the doSJSnt ' 9 ,0 th ® #ource * each document . «"d. » 

iree options and sign at the bottom 

Ths sample sticker shown betowwfflbo 
efflxed to all Level 2B documents 



of the page. 

The sample sticker shown below vibe 
efflxad to ei Level 1 document! 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 



_<=£ 



St» 






TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



The temple sticker ahown bekjwwtnb© 
affixed to al Level 2A documents 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE. AND IN ELECTRONIC MEDIA - 
FOR ERIC COLLECTION SUBSCRIBERS ONLY 
HAS BEEN GRANTED BY 



<=£ 









TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



2A 



Level 1 

(2 



PERMISSION TO REPRODUCE ANO 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE ONLY HAS BEEN GRANTEO BY 






0 



& 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



2B 



Level 2A 
* 

I 

□ 



Level 2B 

□ 



Check here tar Level 1 release. permitting 

ERIC erUOwJi medto (eg., dedionic) endpaper 
copy. 



Check here for Level 2A rate**, permuting 
>*prc)ductton end fls s emtoe t lon In m to o ficf la and in 
alectronfc media for ERIC archival coQection 
eubscrfoem only 



Check here lor Level 2B release, permitting 

reproductkm and dissemination to microfiche only 






Sign 
hen,-* 
please 




indicated above. Reproduction bom the ^Cnba^^^ekct^m^bT^^^^^^ disseminate this document 
contractors requves permission from the oopvriohthoU^^Z^^L tfMOT ERtc om Pio^ and its system 
to safe* (nfomurf^Z^educ^^ 

" pirttfaTjCftl 



2L fp/Uf>,- — mJSSSSl 

$Z- &N Schx.rwo^ (A<x[[ , Cotl&xcP wru Mfctehr, TmltQ 

ER | C agt mzMt W&2?' 1 



t ‘<L 



(over) 




ClearingtiOMse on Assessment and Evaluation 



March 2000 

Dear AERA Presenter, 



University of Maryland 
1129 Shriver Laboratory 
College Park, MD 20742-5701 

Tel: (800) 464-3742 
(301)405-7449 
FAX: (301)405-8134 
ericae @ ericae.net 
http://ericae.net 



Congratulations on being a presenter at AERA. The ERIC Clearinghouse on Assessment and 
Evaluation would like you to contribute to ERIC by providing us with a written copy of your 
presentation. Submitting your paper to ERIC ensures a wider audience by making it available to 
members of the education community who could not attend your session or this year's conference. 

Abstracts of papers accepted by ERIC appear in Resources in Education (R1E) and are announced to over 
5,000 organizations. The inclusion of your work makes it readily available to other researchers, provides a 
permanent archive, and enhances the quality of R1E. Abstracts of your contribution will be accessible 
through the printed, electronic, and internet versions of R1E. The paper will be available full-text, on 
demand through the ERIC Document Reproduction Service and through the microfiche collections 
housed at libraries around the world. 



We are gathering all the papers from the AERA Conference. We will route your paper to the 
appropriate clearinghouse and you will be notified if your paper meets ERIC's criteria. Documents 
are reviewed for contribution to education, timeliness, relevance, methodology, effectiveness of 
presentation, and reproduction quality. You can track our processing of your paper at 

http://ericae.net. 

To disseminate your work through ERIC, you need to sign the reproduction release form on the 
back of this letter and include it with two copies of your paper. You can drop of the copies of 
your paper and reproduction release form at the ERIC booth (223) or mail to our attention at the 
address below. If you have not submitted your 1999 Conference paper please send today or 
drop it off at the booth with a Reproduction Release Form. Please feel free to copy the form 
for future or additional submissions. 



Mail to: AERA 2000/ERIC Acquisitions 

The University of Maryland 
1 129 Shriver Lab 
College Park, MD 20742 



Sincerely, 




Lawrence M. Rudner, Ph.D. 
Director, ERIC/AE 



ERIC/AE is a project of the Department of Measurement, Statistics and Evaluation 
at the College of Education, University of Maryland. 



